# INT8 quantization

This is an INT8-quantized Mistral-Small-3.1-24B-Instruct-2503 model, optimized by Red Hat and Neural Magic, suitable for fast response and low-latency scenarios.

A Mistral instruction fine-tuned model with 24B parameters after INT8 quantization, significantly reducing GPU memory requirements and improving computational throughput.

INT8 quantized version of DeepSeek-R1-Distill-Qwen-32B, reducing VRAM usage and improving computational efficiency through weight and activation quantization.

INT8 quantized version of Qwen2.5-7B-Instruct, suitable for multilingual scenarios in both commercial and research applications, optimized for memory requirements and computational throughput.

The ONNX INT8 quantized version of BAAI/bge-m3, suitable for dense retrieval tasks, and optimizes compatibility with Vespa Embedding.

This is an INT8 PyTorch model statically quantized using Intel® Neural Compressor post-training, based on Google's ViT model fine-tuning, significantly reducing model size while maintaining high accuracy.

I-BERT is a pure integer-quantized version of RoBERTa-large, using INT8 to store parameters and integer operations for inference, achieving up to 4x inference acceleration.